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PROOF ON CONCEPTION 
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LOCATION OF FiRST DESCRIPTION / 
DRAWING 
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B. TO WHOM WAS INVENTION FiRST DISCLOSED? 

Kevin Martin 

DATE 
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4. REDUCTION TO PRACTICE 

A. WAS A DEVICE EMBODYING THE INVENTION 
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DATE STARTED 
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DATE COMPLETED 
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TIME SPENT 
200 hours 
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PRESENT LOCATION OF DEVICE 
HRL BIdg 254, 3G21 
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NOTE: ALL EVIDENCE OF CONCEPTION (FIRST DRA WING AND FIRST WRITTEN DESCRIPTION) AND EVIDENCE OF REDUCTION 
TO PRACTICE (DEVICE EMBODYING THE INVENTION AND TEST DATA) MUST BE RETAINED. 
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YES 
NO X 

CONTRACT NUMBER AND TITLE 
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A. IS THERE A PUBLICATION OR PUBLIC 
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8. ARE THERE ANY RELATED INVENTION 

DISCLOSURES OR PATENT APPLICATIONS? 

YES 
NO X 

DATE 

IDENTIFY PD NO. ETC. 
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C. ARE THERE ANY PROPOSALS OR REPORTS OR 
OTHER DOCUMENTS RELATING TO THIS 
INVENTION 

YES 
NO X 

DATE 

IDENTIFY 

D. HAS THE INVENTION BEEN USED, CHSCUSSED. 
DEMONSTRATED OR OTHERWISE DISCLOSED 
OUTSIDE THE COMPANY {SUCH AS TO A VENDOR 
OR CUSTOMER)? 

YES 
NO X 

DATE 

TO / FOR WHOM (COMPANY / PERSON) 

7. SALE 

A. HAS PRODUCT EMBODYING INVENTION OR MADE 
BY INVENTION BEEN PROPOSED. SOLD, OR 
OFFERED FOR SALE? 

YES □ 
NO X 

ORDER NO. 

ORDER DATE 

DELIVERY DATE 

OATE OFFERED OR 
PROPOSED 

B. IS PRODUCT EMBODYING INVENTION OR MADE BY 
INVENTION IN A DELIVERABLE ITEM? 

YES □ 
NO X 

DELIVERY DATE 



8. SUMMARY OF THE INVENTION 


A. GIVE A BRIEF DESCRIPTION OF YOUR INVENTION, PARTICULARLY POINTING OUT WHAT IS BELIEVED 
TO BE NOVEL (THE "HEART OF WHAT IS NEW). 

This invention describes the system needed to enable mobile users Jo place and access audio and other information at locations in the environment. The 
novel aspect of the system is the unique convergence of technologies that provide a real time pubiicfy (or controlled, privately) accessible "whiteboard" of 
audio and other information types delivered as audio. Both mobile and fixed users can add location specific information to a distributed web-based server 
that is then accessible by users based on a variety of location based queries. The system requires no additional infrastructure such as special devices or 
networks and enables real time location specific annotations and information. Figure 1 shows the overall system concept 



Figure 1 : Real time location embedded audio with no added infrastructure supporting stationary and mobile users. 

B. EXPLAIN THE PURPOSE AND ADVANTAGES OF YOUR INVENTION. (WHAT WILL THE INVENTION DO BETTER 
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AudicOnLocatton enables users to hear audio that is virtually embedded in places with context as they move through the environment. It aJJows the user to 
annotate places with audio in real time, from either fixed or mobile systems, that is then stored in a server and can be accessed remotely from fixed systems 
or as the user moves to new locations or points in the direction of previously annotated objects or places. 

Previous systems surfer from a variety of deficiencies that require external infrastructure to enable user tracking, use very coarse position information (such 
as celfular systems), can not use orientation information, do not enable real time access to new Information, and do not enable users to add new 
information. Our system enables any user to access embedded audio attached there by any other user. It also provides a real time method for placing new 
annotations on places in the environment which previous systems do not allow. 


C. IDENTIFY THE COMPANY PROGRAM OR PRODUCT UNE TO WHICH THE INVENTION APPLIES, AND THE EXPECTED VALUE 
TO THE PROGRAM OR PRODUCT LINE. ALSO IDENTIFY POTENTIAL COMMERCIAL APPLICATION OF THIS INVENTION, 
INCLUDING AUTOMOTIVE APPLICATIONS. IF ANY. 

This system will have direct applicability to soldier technologies including Land Warrior, SUOSAS, Special Forces, and mobile command and control by 
enabling rapid smart environment audio infrastructure for communication and situation awareness. 

In addition, the capability can be applied within automobiles or other moving vehicles to enable smart highway communication with driver. Advanced cell 
phones with location awareness will also take advantage of this system functionality, as will rapid infrastructure to support education, museums, etc. A large 
number of location specific information services are possible with such a system. 


D. IDENTIFY THE PRIOR ART KNOWN TO YOU WHICH IS IMPROVED UPON OR DISPLACED BY YOUR INVENTION. AND STATE 
IN DETAIL. TF KNOWN, THE DISADVANTAGES OF THE CLOSEST PRIOR ART. 

Museums employ an active system with either user initiated audio from tape recorders at known locations or active infrared communication between fixed 
infrastructure and modified tape recorder playback devices. These do not enable widespread use, user annotation, device or user tracking, real time 
updates, or a variety of advanced functions possible with more capable playback using mobile, wearable computing. They also require fixed, static 
hardware and software infrastructure with certain higher maintenance costs. 

AirRash.com will offer a sen/ice for mobile phone users later this year that provides business, entertainment, and travel information specific to a user's 
location. This system is based on a cell phone user's known position location determined by which antenna is used to access the network, allowing search 
results to be tailored to a user's location. It does not provide accurate position information or orientation sensitivity. Some ability to access any geographic 
information is provided through a database service, which could also be used by this invention. 


9. DETAILED DESCRIPTION 

DESCRIBE YOUR INVENTION IN DETAIL, USING NECESSARY ADDITIONAL SHEETS 

A. BE SURE THAT EACH SHEET IS DATED, AND SIGNED BY EACH INVENTOR AND TWO WITNESSES. 

(HRL FORM 236°* CS SHOULD BE USED. IF PRACTICAL). 
8. ATTACH COPIES OF DRAWINGS OR DETAILED REPORTS HELPFUL IN UNDERSTANDING HOW YOUR INVENTION WORKS 


C. IF YOUR INVENTION HAS BEEN TESTED, BRIEFLY SUMMARIZE THE TEST RESULTS WHICH CONFIRM THE 
FUNCTIONS AND ADVANTAGES LISTED IN 8 D ABOVE. 

AudioOnLocation is a system that enables mobile users to obtain audio information that is context and location dependent. It also provides the necessary 
infrastructure to enable any user at any location to embed audio in specif ic locations that can then be accessed by other users either from that location 
usrng special information access appliances, or from any remote location using a variety of interfaces through the World Wide Web. Figure 2 below shows 
me sequence of operations (and optional operations) that occur when the system is used to retrieve audio either via explicit or implicit (push) request 
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Verbal Request(opt) Speech-to-Text (opt) -►Position Tag -> Compress (opt) -> Transmit (opt) 


Client 


Play Spatialize (opt) 


-Tcxt-to-Speech (opt) 4— Decompress (opt) 
Figure 2: Operations performed in audio retrieval. 



Client or Server 


remote locations and be connected via wireless or cellular networks to the client: 

The system may be used in an information push mode where user location is constantly streamed to ^j^^J^^^ the server 
auton^ticallyWentifies and sends the audio to the client based on predetermined parameters such as distance from current pos.ton. 

A mom advanced use of AudtoOnLocation enables explicit querying of the Information Server based on position and orientation of the user's hand held or 
b£?£j£K object orregion of interest from a remote position. The system determines what objects are 

potentially of interest using the user's current position and the pointing vector and retneves and plays them. 
The AudioOnLocation Information Server is composed of the following software functions (shown graphically in Figure 3): 

1 ^ User Interface* This function provides the user with the ability to selectively control the scope and generation of queries using voice, keyboard, touch 
' paTorTS to provide relevant information in the form of preferences to aid in selectmg and controlling the audio 

playback messages. The user may explicitly provide a position. 

2) Position Sensors: A variety of complementary sensors provide the user's position and, optionally, orientation (a six dimensional vector) to the position 
manager. 

3) Content Manager: This function receives content from the User Interface and sends links to content to the Database Update Manager. It sends the 
content (file) to the content pool. These functions are separated to enable distributed management and storage of the content. 

4) Position Manager: This function takes any user supplied position data or alternatively, position data from sensors and sends it to the database update 
manager to be linked with the current recorded content, and to the database query manager. 
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5) Database Update Manager: This function matches the content reference and the position data and updates the database. 

infS™ S Private, business (e.g. restaurant, commerce), nav.gat.on. and general. 

7, Playback Manager. This h^c* ^ 

from wnwPSSK . ZTpX playback Matures, network or cellular connectivity constants, etc. 
9) Content Pooh The audio, text, or other information is stored as files referenced by URL and may be local or remote (completely distributed) from the 

other system components. 



Figure 3: Information Server Block Diagram 


in one preferred embodiment of the invention, a user with a mobile phone and GPS (such 'as the Garmin NavTalk product) dials f e "un^r tor the 
AudioOnlLocation Information Server. Figure 4 shows the user's interaction with the Information Server through an automatic call answering service for the 
simplest case in which there is only one item of information within the desired range of the user's position. 

Figure 5 shows a slightly more complex case in which the user hears a menu of items that allow selection of specific information types or system functions. 
In this case, the user selects to hear ail the current audio and might hear something like There are 5 information items within 20 meters ot your current 
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oosition You may choose to listen to any one of them by entering the number of the Hem followed by the # sign." The user enters a number for the item 
wWch Ts rlwevKm^fe server and played through the cell phone taw as analog audK). 

FigureSshowsacasewhereauser chooses fromamenu of Hems that allows the user to speaK an audio annotation at the user's current pos.t.on that ,s 
placed in the AudioOnLocation Information Server. 



User DialsNunibcr 

jSvstem Requests Position_ 

Cell Phone Responds 
W/position 

User Provides Position 
. Audio 


User Position flat, long) 


Audio _ 



Figure 4: A simple cell phone based use of AudioOnLocation.. 



User Dials Number 


^Svstem Requests Position I 



Cell Phone Responds 


W/position 


User Provides Position 


^ Menu of Choices 




User Selects from Menu 


Audio 




Figure 5: A menu-based interaction between user with cell phone and AudioOnLocation system. 
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User Dials Number 




^System Requests Position 




Cell Phone Responds 


W/position 


User Provides Position 


^ ^ Menu of Choices 



> 

User Selects from Menu 


User Speaks Audio 




User Position (lat, long) 


Audio Annotation 



Figure 6: A user adds audio annotation in the information server at the current position. 
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